Understanding receiver operating characteristic (ROC) curves.

In this issue of the Journal, Auer and colleagues conclude that serum levels of neuron-specific enolase (NSE), a biochemical marker of ischemic brain injury, may have clinical utility for the prediction of survival to hospital discharge in patients experiencing the return of spontaneous circulation following at least 5 minutes of cardiopulmonary resuscitation. The authors used a receiver operating characteristic (ROC) curve to illustrate and evaluate the diagnostic (prognostic) performance of NSE. We explain ROC curve analysis in the following paragraphs. The term “receiver operating characteristic” came from tests of the ability of World War II radar operators to determine whether a blip on the radar screen represented an object (signal) or noise. The science of “signal detection theory” was later applied to diagnostic medicine. The determination of an “ideal” cut-off value is almost always a trade-off between sensitivity (true positives) and specificity (true negatives). As both change with each “cut-off” value it becomes difficult for the reader to imagine which cut-off is ideal. The ROC curve offers a graphical illustration of these trade-offs at each “cut-off” for any diagnostic test that uses a continuous variable. Ideally, the best “cutoff” value provides both the highest sensitivity and the highest specificity, easily located on the ROC curve by finding the highest point on the vertical axis and the furthest to the left on the horizontal axis (upper left corner) (Fig. 1). However, it is rare that this ideal can be achieved, so that, for example, one may opt to choose a higher sensitivity at the cost of lower specificity. In the NSE study, the authors chose a cut-off point of >30 μg/L with a specificity of 100% and sensitivity of 79% (Fig. 2). A cut-off point with high specificity allows the authors to “rule-in” the outcome for all patients with a NSE value above the selected cutoff. The study indicates that patients with a NSE level >30 μg/L will die before hospital discharge and those with a NSE level <29 μg/L will possibly survive to hospital discharge. The area under the ROC curve (AUC) is widely recognized as the measure of a diagnotic test’s discriminatory power. The maximum value for the AUC is 1.0, thereby indicating a (theoretically) perfect test (i.e., 100% sensitive